1. Field of the Invention
This invention relates generally to hardware and methods for rapid, low-latency detection of viruses in network transmissions and specifically to methods for determining which parts of a network transmission might contain viruses and checking and cleaning only those parts of the transmission which could potentially be infected as soon as sufficient information is available for such a check. In particular, the present invention is directed to a system and method in which scanned data will be output as soon as possible as well to complete both the input and output aspects of streams
2. Description of the Related Art
The first generation of computer viruses appeared in the 1980's. These early viruses had little effect because they typically were spread using removable media such as floppy disks. For instance, when an infected floppy disk was inserted into a clean computer, the floppy disk would infect the machine by embedding a virus into the memory and/or storage of the machine and then the virus would copy itself to every uninfected floppy disk inserted from that time forward. Even when the machine was rebooted the machine would often remain infected because of the likelihood that the boot system disk had also become infected. Of course, computers with hard disks were permanently infected unless the virus was discovered and removed. All computers that came in contact with removable media from an infected computer and that were running the same operating system would also become carriers of that particular virus. Of course, viruses that moved from computer to computer by human hands were replicated slowly, so the effects of such floppy-transmitted viruses were limited.
In the 1990's many computers began to be connected to the Internet and people found it very convenient to use the Internet as a communication means. While the Internet improved connectivity and communication between computers, it also gave virus creators a new, fast and effective transport medium. As a result, almost all current viruses attempt to replicate themselves via a network using email, the worldwide web, and other network transfer protocols. MyDoom and SoBig are examples of such network borne viruses.
There are several companies, including McAfee, Inc. of Santa Clara, Calif., USA that provide desktop anti-virus protection. Desktop tools are tools that can search through files on hard disks connected to a computer to detect and remove viruses. The tools available from these companies work on individual computers to eliminate viruses by looking through file storage or disks one file at a time to determine if any of those files have code in them that matches the signature of a known virus or in other ways appear to be a virus. If a virus is detected, the software either deletes the file or the part of the file that contains the virus, rendering it ineffective. This strategy for finding and removing viruses is well known in the art and has been used from the days of floppy-disk borne viruses
More advanced forms of finding and removing viruses exist today. Many companies install gateways or computers that sit between a company's internal network or Local Area Network (LAN) and the Internet or Wide Area Network (WAN). These gateways are also called firewalls or routers, depending on their function. Gateways provide services that protect computers on the Local Area Network from access by other computers outside the LAN. Since gateways control all Internet access (in other words, all traffic between the LAN and WAN must travel through such a gateway) a gateway is in a unique position as a computer that can check all files moving across the LAN boundary—the path that most viruses take today.
F-Secure Corporation of Helsinki, Finland provides a tool that looks through files stored on a gateway. F-Secure's product looks through each file that is received once it is stored on the hard disk of the gateway. For instance, email destined for a computer on the LAN is temporarily stored on a gateway as it is queued up for the recipient inside the network. F-Secure's product will scan such email and any attachments to determine whether or not they contain viruses. If they do, the email can be cleaned and either forwarded to the intended recipients or rejected and sent back to the senders.
Unfortunately, F-Secure's email scanning technique slows down the network. Each file requested by an internal computer or sent by an external computer must be stored locally on the gateway and then searched for viruses. When a set of compressed files, like a ZIP file is received, the entire set of files is stored and checked individually before any of the files is transmitted to the requesting host. This can cause unacceptable delays to the requesting host, including time-outs that prevent the requesting host from getting any data at all. Time delays associated with network transmission are referred to as “latency,” and most computer network protocols have time-outs that limit the amount of latency they can accept. Long delays can cause a program or system to stop working because it incorrectly detects that it is no longer connected to the network.
All current virus checking or scanning software uses the file interface to the computer system to check for viruses. The file API in most current operating systems is exactly the same, allowing any program to open, read, or write to a file using a set of basic system calls. Files can be accessed randomly—allowing direct access to a specific part of the file at any time. In fact, file access even works across networks on different operating systems. An Apple PowerBook computer can access files stored on a Windows computer or Linux system because there exists file access protocols (for instance, SMB) which allow a program on one computer to freely access files on another computer without knowing that the file is being accessed over the network. Any program, including virus check programs, that are written to detect and clean files can use any files accessible to the computer without regard to the actual storage location of the bits. The easiest way to create a gateway-based virus-scanning tool is to store incoming files temporarily on the system and run the file-based virus scanner against the file.
What is needed is a method of reducing the latency of virus checking on a gateway system so that large packets of information are not unnecessarily delayed during a virus check. What is further needed is a method of checking just part of a transmission for a virus before the entire transmission is completed. What is further needed in a system for checking for viruses on a gateway, are methods of deciding which parts of a data transmission must be checked for viruses to avoid unnecessary delays in data transmission.